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AMENDMENTS TO THE CLAIMS 

1 . (Currently amended) A system for generating creating a description record from 
multimedia information, comprising: 

(a) at least one multimedia information input interface receiving said 
multimedia information; 

(b) a computer processor, coupled to said at least one multimedia information 
input interface, receiving said multimedia information therefrom, processing said 
multimedia information by performing object extraction processing to g e nerate create 
multimedia object descriptions from said multimedia information, and processing said 
generated created multimedia object descriptions by object hierarchy processing to 
generate create multimedia object hierarchy descriptions indicative of an organization of 
said object descriptions, wherein at least one description record including said multimedia 
object descriptions and said multimedia object hierarchy descriptions is gen e rated created 
for content embedded within said multimedia information; and 

(c) a data storage system, operatively coupled to said processor, for storing said at 
least one description record. 

2. (Original) The system of claim 1, wherein said multimedia information 
comprises image information, said multimedia object descriptions comprise image object 
descriptions, and said multimedia object hierarchy descriptions comprise image object 
hierarchy descriptions. 

3. (Currently amended) The system of claim 2, wherein said object extraction 
processing comprises: 
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(a) image segmentation processing to segment each image in said image 
information into regions within said image; and 

(b) feature extraction processing to generate create one or more feature descriptions 
for one or more of said regions; 

whereby said generated created object descriptions comprise said one or more 
feature descriptions for one or more of said regions. 

4. (Original) The system of claim 3, wherein said one or more feature descriptions 
are selected from the group consisting of text annotations, color, texture, shape, size, and 
position. 

5. (Currently amended) The system of claim 2, wherein said object hierarchy 
processing comprises physical object hierarchy organization to g e nerat e create physical 
object hierarchy descriptions of said image object descriptions that are based on spatial 
characteristics of said objects, such that said image object hierarchy descriptions comprise 
physical descriptions. 

6. (Currently amended) The system of claim 5, wherein said object hierarchy 
processing further comprises logical object hierarchy organization to gen e rat e create 
logical object hierarchy descriptions of said image object descriptions that are based on 
semantic characteristics of said objects, such that said image object hierarchy descriptions 
comprise both physical and logical descriptions. 
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7. (Currently amended) The system of claim 6, wherein said object extraction 
processing comprises: 

(a) image segmentation processing to segment each image in said image 
information into regions within said image; and 

(b) feature extraction processing to g e n e rat e create object descriptions for one or 
more of said r e gion regions ; 

and wherein said physical hierarchy organization and said logical hierarchy 
organization generate create hierarchy descriptions of said object descriptions for said one 
or more of said regions. 

8. (Original)The system of claim 7, further comprising an encoder receiving said 
image object hierarchy descriptions and said image object descriptions, and encoding said 
image object hierarchy descriptions and said image object descriptions into encoded 
description information, wherein said data storage system is operative to store said 
encoded description information as said at least one description record. 

9. (Original) The system of claim 1, wherein said multimedia information 
comprises video information, said multimedia object descriptions comprise video object 
descriptions including both event descriptions and object descriptions, and said multimedia 
hierarchy descriptions comprise video object hierarchy descriptions including both event 
hierarchy descriptions and object hierarchy descriptions. 
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10. (Currently amended) The system of claim 9, wherein said object extraction 
processing comprises: 

(a) temporal video segmentation processing to temporally segment said video 
information into one or more video events or groups of video events and generate create 
event descriptions for said video events, 

(b) video obj ect extraction processing to segment said one or more video events 
or groups of video events into one or more regions, and to generat e create object 
descriptions for said regions; and 

(c) feature extraction processing to generat e create one or more event feature 
descriptions for said one or more video events or groups of video events, and one or more 
object feature descriptions for said one or more regions; 

wherein said generated created video object descriptions include said event feature 
descriptions and said object descriptions. 

11. (Original) The system of claim 10, wherein said one or more event feature 
descriptions are selected from the group consisting of text annotations, shot transition, 
camera motion, time and key frame, and wherein said one or more object feature 
descriptions are selected from the group consisting of color, texture, shape, size, position, 
motion, and time. 

12. (Currently amended) The system of claim 9, wherein said object hierarchy 
processing comprises physical event hierarchy organization to generat e create physical 
event hierarchy descriptions of said video object descriptions that are based on temporal 
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characteristics of said video objects, such that said video hierarchy descriptions comprise 
temporal descriptions. 

13. (Currently amended) The system of claim 12, wherein said object hierarchy 
processing further comprises logical event hierarchy organization to g e nerat e create logical 
event hierarchy descriptions of said video object descriptions that are based on semantic 
characteristics of said video objects, such that said hierarchy descriptions comprise both 
temporal and logical descriptions. 

14. (Currently amended) The system of claim 13, wherein said object hierarchy 
processing further comprises physical and logical object hierarchy extraction processing, 
receiving said temporal and logical descriptions and g e nerating creating object hierarchy 
descriptions for video objects embedded within said video information, such that said 
video hierarchy descriptions comprise temporal and logical event and object descriptions. 

15. (Currently amended) The system of claim 14, wherein said object extraction 
processing comprises: 

(a) temporal video segmentation processing to temporally segment said video 
information into one or more video events or groups of video events and generate create 
event descriptions for said video events, 

(b) video object extraction processing to segment said one or more video events or 
groups of video events into one or more regions, and to generat e create object descriptions 
for said regions; and 
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(c) feature extraction processing to generat e create one or more event feature 
descriptions for said one or more video events or groups of video events, and one or more 
object feature descriptions for said one or more regions; 

wherein said generated created video object descriptions include said event feature 
descriptions and said object descriptions, and wherein said physical event hierarchy 
organization and said logical event hierarchy organization generat e create hierarchy 
descriptions from said event feature descriptions, and wherein said physical object 
hierarchy organization and said logical object hierarchy organization gen e rat e create 
hierarchy descriptions from said object feature descriptions. 

16. (Original) The system of claim 15, further comprising an encoder receiving said 
video object hierarchy descriptions and said video object descriptions, and encoding said 
video object hierarchy descriptions and said video object descriptions into encoded 
description information, wherein said data storage system is operative to store said 
encoded description information as said at least one description record. 

17. (Currently amended) A method for gen e rating creatinfi a description record 
from multimedia information, comprising the steps of: 

(a) receiving said multimedia information; 

(b) processing said multimedia information by performing object extraction 
processing to generate create multimedia object descriptions from said multimedia 
information; 

(c) processing said generated created multimedia object descriptions by object 
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hierarchy processing to gen e rate create multimedia object hierarchy descriptions indicative 
of an organization of said object descriptions, wherein at least one description record 
including said multimedia object descriptions and said multimedia object hierarchy 
descriptions is gen e rated created for content embedded within said multimedia information; 
and 

(d) storing said at least one description record. 

18. (Original) The method of claim 17, wherein said multimedia information 
comprises image information, said multimedia object descriptions comprise image object 
descriptions, and said multimedia object hierarchy descriptions comprise image object 
hierarchy descriptions. 

19. (Currently amended) The method of claim 18, wherein said object extraction 
processing step comprises the sub-steps of: 

(a) image segmentation processing to segment each image in said image 
information into regions within said image; and 

(b) feature extraction processing to g e nerate create one or more feature descriptions 
for one or more of said regions; 

whereby said g e n e rated created image object descriptions comprise said one or 
more feature descriptions for one or more of said regions. 
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20. (Original) The method of claim 19, wherein said one or more feature 
descriptions are selected from the group consisting of text annotations, color, texture, 
shape, size, and position. 

21 . (Currently amended) The method of claim 18, wherein said step of object 
hierarchy processing includes the sub-step of physical object hierarchy organization to 
generat e create physical object hierarchy descriptions of said image object descriptions 
that are based on spatial characteristics of said objects, such that said image hierarchy 
descriptions comprise physical descriptions. 

22. (Currently amended) The method of claim 21, said step of object hierarchy 
processing further includes the sub-step of logical object hierarchy organization to 
generate create logical object hierarchy descriptions of said image object descriptions that 
are based on semantic characteristics of said objects, such that said image object hierarchy 
descriptions comprise both physical and logical descriptions. 

23. (Currently amended) The method of claim 22, wherein said step of object 
extraction processing further includes the sub-steps of: 

(a) image segmentation processing to segment each image in said image 
information into regions within said image; and 

(b) feature extraction processing to generate create object descriptions for one or 
more of said region; 
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and wherein said physical object hierarchy organization sub-step and said logical 
object hierarchy organization sub-step generat e create hierarchy descriptions of said object 
descriptions for said one or more of said regions. 

24. (Previously presented) The method of claim 18, further comprising the step of 
encoding said image object descriptions and said image object hierarchy descriptions into 
encoded description information prior to said data storage step. 

25. (Original) The method of claim 17, wherein said multimedia information 
comprises video information, said multimedia object descriptions comprise video object 
descriptions including both event descriptions and object descriptions, and said multimedia 
hierarchy descriptions comprise video object hierarchy descriptions including both event 
hierarchy descriptions and object hierarchy descriptions. 

26. (Currently amended) The method of claim 25, wherein said step of object 
extraction processing comprises the sub-steps of: 

(a) temporal video segmentation processing to temporally segment said video 
information into one or more video events or groups of video events and gen e rate create 
event descriptions for said video events, 

(b) video object extraction processing to segment said one or more video events or 
groups of video events into one or more regions, and to generate create object descriptions 
for said regions; and 
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(c) feature extraction processing to g e nerate create one or more event feature 
descriptions for said one or more video events or groups of video events, and one or more 
object feature descriptions for said one or more regions; 

wherein said generated created video object descriptions include said event feature 
descriptions and said object descriptions. 

27. (Original) The method of claim 26, wherein said one or more event feature 
descriptions are selected from the group consisting of text annotations, shot transition, 
camera motion, time and key frame, and wherein said one or more object feature 
descriptions are selected from the group consisting of color, texture, shape, size, position, 
motion, and time. 

28. (Currently amended) The method of claim 25, wherein said step of object 
hierarchy processing includes the sub-step of physical event hierarchy organization to 
generate create physical event hierarchy descriptions of said video object descriptions that 
are based on temporal characteristics of said video objects, such that said video hierarchy 
descriptions comprise temporal descriptions. 

29. (Currently amended) The method of claim 28, wherein said step of object 
hierarchy processing further includes the sub-step of logical event hierarchy organization 
to g e n e rate create logical event hierarchy descriptions of said video object descriptions that 
are based on semantic characteristics of said video objects, such that said hierarchy 
descriptions comprise both temporal and logical descriptions. 
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30. (Currently amended) The method of claim 29, wherein said step of object 
hierarchy processing further comprises the sub-step physical and logical object hierarchy 
extraction processing, receiving said temporal and logical descriptions and generating 
creating object hierarchy descriptions for video objects embedded within said video 
information, such that said video hierarchy descriptions comprise temporal and logical 
event and object descriptions.. 

31. (Currently amended) The method of claim 30, wherein said step of object 
extraction processing comprises the sub-steps of: 

(a) temporal video segmentation processing to temporally segment said video 
information into one or more video events or groups of video events and generat e create 
event descriptions for said video events, 

(b) video object extraction processing to segment said one or more video events or 
groups of video events into one or more regions, and to gen e rat e create object descriptions 
for said regions; and 

(c) feature extraction processing to g e nerate create one or more event feature 
descriptions for said one or more video events or groups of video events, and one or more 
object feature descriptions for said one or more regions; 

wherein said gen e rated created video object descriptions include said event feature 
descriptions and said object descriptions, and wherein said physical event hierarchy 
organization and said logical event hierarchy organization g e nerat e create hierarchy 
descriptions from said event feature descriptions, and wherein said physical object 
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hierarchy organization and said logical object hierarchy organization generate create 
hierarchy descriptions from said object feature descriptions. 

32. (Previously presented) The method of claim 31, further comprising the step of 
encoding said video object descriptions and said video object hierarchy descriptions into 
encoded description information prior to said data storage step. 

33. (Currently amended) A computer readable media containing digital information 
with at least one multimedia description record describing multimedia content for 
corresponding multimedia information, the description record comprising: 

(a) one or more multimedia object descriptions, generat e d created by 
performing object extraction processing, said object descriptions describing 
corresponding multimedia objects; 

(b) one or more features characterizing each of said multimedia object 
descriptions; and 

(c) one or more multimedia object hierarchy descriptions indicative of an 
organization of said object descriptions, if any, relating at least a portion of 
said one or more multimedia objects in accordance with one or more 
characteristics. 

34. (Original) The computer readable media of claim 33, wherein said multimedia 
information comprises image information, said multimedia objects comprise image 
objects, said multimedia object descriptions comprise image object descriptions, and said 
multimedia object hierarchy descriptions comprise image object hierarchy descriptions. 
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35. (Original) The computer readable media of claim 34, wherein said one or more 
features are selected from the group consisting of text annotations, color, texture, shape, 
size, and position. 

36. (Original) The computer readable media of claim 34, wherein said image object 
hierarchy descriptions comprise physical object hierarchy descriptions of said image object 
descriptions based on spatial characteristics of said image objects. 

37. (Original) The computer readable media of claim 36, wherein said image object 
hierarchy descriptions further comprises logical object hierarchy descriptions of said 
image object descriptions based on semantic characteristics of said image objects. 

38. (Original) The computer readable media of claim 33, wherein said multimedia 
information comprises video information, said multimedia objects comprise events and 
video objects, said multimedia object descriptions comprise video object descriptions 
including both event descriptions and object descriptions, said features comprise video 
event features and video object features, and said multimedia hierarchy descriptions 
comprise video object hierarchy descriptions including both event hierarchy descriptions 
and object hierarchy descriptions. 

39. (Original) The computer readable media of claim 38, wherein said one or more 
event feature descriptions are selected from the group consisting of text annotations, shot 
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transition, camera motion, time and key frame, and wherein said one or more object feature 
descriptions are selected from the group consisting of color, texture, shape, size, position, 
motion, and time. 

40. (Original) The computer readable media of claim 38, wherein said event 
hierarchy descriptions comprise one or more physical hierarchy descriptions of said events 
based on temporal characteristics. 

41. (Original) The computer readable media of claim 40, wherein said event 
hierarchy descriptions further comprise one or more logical hierarchy descriptions of said 
events based on semantic characteristics. 

42. (Original) The computer readable media of claim 38, wherein said object 
hierarchy descriptions comprise one or more physical hierarchy descriptions of said 
objects based on temporal characteristics. 

43. (Original) The computer readable media of claim 39, wherein said object 
hierarchy descriptions further comprise one or more logical hierarchy descriptions of said 
objects based on semantic characteristics. 
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